Most people get to Victor Harbor by a 90 minute or so drive from Adelaide, but there is a coach service run by Premier Stateliner (http://www.premierstateli...) from Adelaide Central Bus Station and Goolwa. Fares are $19.50/$9.50. You can switch to a airport bus at Marion Shopping Centre rather than in Adelaide City, but check the timetable as buses to Marion only go every hour or so.
The Steamranger runs the Southern Encounter (http://members.iinet.net....) service on Sundays twice monthlyDeparting from Mt Barker in the Adelaide Hills, giving around 4 hours to explore the town. Adults $48.00. Children 5 years and over $26.00. They also run a train to Goolwa Sundays, Wednesdays and every day during the school holidays. One way fares are available on request.